Feature selection for high-dimensional classification using a competitive swarm optimizer

When solving many machine learning problems such as classification, there exists a large number of input features. However, not all features are relevant for solving the problem, and sometimes, including irrelevant features may deteriorate the learning performance. Therefore, it is essential to select the most relevant features, which is known as feature selection. Many feature selection algorithms have been developed, including evolutionary algorithms or particle swarm optimization (PSO) algorithms, to find a subset of the most important features for accomplishing a particular machine learning task. However, the traditional PSO does not perform well for large scale optimization problems, which degrades the effectiveness of PSO for feature selection when the number of features dramatically increases. In this paper, we propose to use a very recent PSO variant, known as competitive swarm optimizer (CSO) that was dedicated to large-scale optimization, for solving high-dimensional feature selection problems. In addition, the CSO, which was originally developed for continuous optimization, is adapted to performing feature selection that can be considered as a combinatorial optimization problem. An archive technique is also introduced to reduce computational cost. Experiments on six benchmark datasets demonstrate that compared to the canonical PSO based and a state-of-the-art PSO variant for feature selection, the proposed CSO-based Shenkai Gu Department of Computer Science, University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom Ran Cheng Department of Computer Science, University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom Yaochu Jin 1 Department of Computer Science, University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom 2 School of Management Science and Engineering, Dalian University of Technology, Dalian, China; 116023 E-mail: [email protected]